Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Go: New File System Access Sinks #14064

Merged
merged 22 commits into from
Oct 11, 2023

Conversation

am0o0
Copy link
Contributor

@am0o0 am0o0 commented Aug 27, 2023

I added new important System File Access Sinks.
I didn't know how to complete the Library Tests.
Also I put all the things in one query file, I knew that there was frameworks for some of these new sinks but I wanted to keep this pull request simple for review, I think It'll be better if I wait for your suggestion about move each new sink to a proper location.
Please to Skip writing your own local tests refer to following: https://github.com/amammad/afero_And_WebServers

@am0o0 am0o0 requested a review from a team as a code owner August 27, 2023 13:58
@github-actions github-actions bot added the Go label Aug 27, 2023
@ghsecuritylab ghsecuritylab marked this pull request as draft August 27, 2023 16:02
@am0o0 am0o0 changed the title v1 [Go]: New File System Access Sinks Aug 28, 2023
@am0o0 am0o0 changed the title [Go]: New File System Access Sinks Go: New File System Access Sinks Aug 28, 2023
@ghsecuritylab ghsecuritylab marked this pull request as ready for review September 4, 2023 09:00
@am0o0
Copy link
Contributor Author

am0o0 commented Sep 15, 2023

Hi, I just added some comments so there aren't new updates. for better review quality.
Besides The fasthttp already is modeled in its library now, should I remove the fasthttp class from this pull request now?

Copy link
Contributor

@owen-mc owen-mc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for this contribution. I've suggested some changes to make it more consistent with the way the rest of our code is written.

go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Fixed Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Fixed Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
@owen-mc
Copy link
Contributor

owen-mc commented Sep 22, 2023

You also need to add a change note (details here). And the tests are failing because they aren't building correctly - for some reason it can't find the vendored dependencies. I'll look into it more next week.

@am0o0
Copy link
Contributor Author

am0o0 commented Sep 22, 2023

@owen-mc thanks for helping me to improve the quality of my pull request. how much time do usually I have to fix and submit the suggestion generally in the codeql repository?

@owen-mc
Copy link
Contributor

owen-mc commented Sep 22, 2023

@amammad There is no great hurry. If you take more than a month I might ask if you plan to do it at some point or not. I do recommend not leaving them too long, when possible, because sometimes other changes get merged in which cause merge conflicts or require some changes to the PR.

@owen-mc
Copy link
Contributor

owen-mc commented Sep 22, 2023

I've looked into why the test is failing. You need to make stubs for New, Compression, Context, in github.com/kataras/iris/v12. And similarly you need to stub more things for some of the other libraries too. If you run go build . in the test directory then you will get error messages which tell you what's missing.

@am0o0
Copy link
Contributor Author

am0o0 commented Sep 25, 2023

@owen-mc Hi, I've created an Afero module that supports sanitizer now!
Also, I need to put the additional taint step in the related query file( like the path traversal query).

Copy link
Contributor

@owen-mc owen-mc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing all my review comments. The tests would be much better if they were rewritten using InlineExpectationsTest. See here and here for an example. This makes it much easier to read tests because the expected output is contained in comments in the test file itself.

go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Fixed Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
Copy link
Contributor

@owen-mc owen-mc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking really good. Just a few small comments. Also, I'm afraid to say that these models should be moved into different files, one for each library. There are already files for Beego, Gin and Echo. You can make ones for the others. (There is one for Fiber in the experimental folder, which you can ignore.) The tests will have to be split and moved as well.

go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/security/FileSystemAccess.qll Outdated Show resolved Hide resolved
Copy link
Contributor

@owen-mc owen-mc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can put JWT.qll in go/ql/src/experimental/frameworks/JWT.qll and import it as import experimental.frameworks.JWT.

Can I just double check that the additional flow steps are valid for any query, not just this one? The way you’ve done it is fine. When this query is promoted they will probably be changed to yml models, but those can’t be used for experimental queries.

I haven't reviewed the tests yet. I will do that when I can find time.

go/ql/lib/go.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/frameworks/Iris.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/frameworks/Fiber.qll Show resolved Hide resolved
go/ql/lib/semmle/go/frameworks/Iris.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/frameworks/Iris.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/frameworks/Echo.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/frameworks/Afero.qll Outdated Show resolved Hide resolved
@owen-mc
Copy link
Contributor

owen-mc commented Oct 3, 2023

The following files need to be reformatted:

./ql/test/library-tests/semmle/go/frameworks/Beego/test.go ./ql/test/library-tests/semmle/go/frameworks/Echo/test.go ./ql/test/library-tests/semmle/go/frameworks/Afero/main.go ./ql/test/library-tests/semmle/go/frameworks/Fiber/test.go ./ql/test/library-tests/semmle/go/frameworks/Iris/test.go

I'm not sure if you can see the output of CI, but the formatting check wasn't printing which files need to be reformatted, so I have made a PR to fix that.

@am0o0
Copy link
Contributor Author

am0o0 commented Oct 4, 2023

@owen-mc I'm so sorry for many mistakes!
it seems that there were some mistakes that I don't know why I couldn't notice, I think the go build command error was my problem which I didn't notice after running the codeql test too.

@owen-mc
Copy link
Contributor

owen-mc commented Oct 4, 2023

No problem. There is one compile error left for Beego:

| test.go:46:23:46:23 | undefined: beego.BeegoOutput |

Copy link
Contributor

@owen-mc owen-mc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's beginning to look really good. Just a few more comments.

go/ql/lib/semmle/go/frameworks/Afero.qll Outdated Show resolved Hide resolved
go/ql/lib/semmle/go/frameworks/Afero.qll Outdated Show resolved Hide resolved
go/ql/test/library-tests/semmle/go/frameworks/Gin/Gin Outdated Show resolved Hide resolved
Copy link
Contributor

@owen-mc owen-mc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution!

@owen-mc owen-mc merged commit 477d8f8 into github:main Oct 11, 2023
10 checks passed
@am0o0 am0o0 deleted the amammad-go-NewFileSystemAccess branch September 14, 2024 11:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants